AI Models Integration
This document provides detailed information about how FF-API-External integrates with AI services (OpenAI's ChatGPT and Anthropic's Claude) and implements various functionalities related to AI-powered text generation, product matching, and data extraction.
Table of Contents
OpenAI (ChatGPT) Integration
OpenAI Overview
FF-API-External integrates with OpenAI's API to provide advanced language processing capabilities via the ChatGPT model. This integration enables functionalities like product matching, product specification extraction, entity keyword metadata generation, and more.
The implementation is located in models/ai/openai/chatgpt.py and provides a wrapper around OpenAI's API with specialized methods for various use cases.
OpenAI Implementation
The ChatGPT class provides a comprehensive interface for interacting with OpenAI's API:
class ChatGPT:
def __init__(self):
self.api_key = config('CHAT_GPT_API_KEY')
self.openai = openai.api_key = self.api_key
self.client = OpenAI(
api_key=self.api_key,
)
The class uses a retry mechanism to handle API rate limits:
def retry_with_backoff(func):
def wrapper(*args, **kwargs):
retries = 3
for i in range(retries):
try:
return func(*args, **kwargs)
except RateLimitError:
wait_time = 2 ** i # Exponential backoff
logging.warning(f"Rate limit hit. Retrying in {wait_time} seconds...")
time.sleep(wait_time)
except APIError as e:
logging.error(f"API error occurred: {str(e)}")
time.sleep(2 ** i) # Backoff on general API errors too
except Exception as e:
logging.error(f"Unexpected error occurred: {str(e)}")
break
raise Exception("Operation failed after retries")
return wrapper
OpenAI Features
The ChatGPT class provides various specialized methods:
General Text Generation
@retry_with_backoff
def generate_response(self, prompt, full=True, model="gpt-4o", temperature=0.7, max_tokens=150, top_p=1.0,
frequency_penalty=0.0, presence_penalty=0.0):
"""
Generates a response from the model based on the provided prompt and parameters.
"""
Product Matching
@retry_with_backoff
def get_matched_products(self, prompt, etypes=None, full=True, model="gpt-4o-mini", temperature=0.7, max_tokens=10000,
top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0):
"""
Generates product matches based on the provided prompt and parameters.
"""
Product Specifications
@retry_with_backoff
def get_product_specs(self, prompt, full=True, model="gpt-4o", temperature=0.7, max_tokens=2000,
top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0):
"""
Generates detailed product specifications based on the provided prompt.
"""
Entity Keywords Metadata
@retry_with_backoff
def get_entity_keywords_metadata(self, entity, full=True, model="gpt-4o", temperature=0.7, max_tokens=4096,
top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0):
"""
Extracts keywords and metadata from an entity listing.
"""
Entity Accolades Metadata
@retry_with_backoff
def get_entity_accolades_metadata(self, entity, full=True, model="gpt-4o", temperature=0.7, max_tokens=4096,
top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0):
"""
Extracts accolades and citation details from an entity.
"""
Color Matching
@retry_with_backoff
def get_basic_color_matches_from_marketing_color(self, color_name, model="gpt-4o", temperature=0.5, max_tokens=512,
top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0,
full=False):
"""
Matches a marketing color name to basic standard colors.
"""
Image Analysis
@retry_with_backoff
def get_image_description(self, image_url, full=True, model="gpt-4o"):
"""
Generates descriptions of an image at multiple levels of detail.
"""
@retry_with_backoff
def extract_image_colors(self, image_url, full=True, model="gpt-4o"):
"""
Extracts up to 10 main colors from an image.
"""
OpenAI Error Handling
The implementation includes robust error handling with retries and exponential backoff for rate limiting and API errors.
Anthropic (Claude) Integration
Claude Overview
FF-API-External integrates with Anthropic's Claude API to provide additional AI capabilities. The implementation is focused on providing high-quality results for specific use cases like product matching, product specifications extraction, vehicle specifications, and more.
The implementation is located in models/ai/anthropic/claude.py and provides a wrapper around Anthropic's API.
Claude Implementation
The ClaudeAI class provides an interface for interacting with Anthropic's Claude API:
class ClaudeAI:
"""A wrapper class for interacting with Anthropic's Claude AI models."""
VALID_MODELS = {
"claude-3-opus",
"claude-3-sonnet",
"claude-3.5-sonnet",
"claude-3-haiku"
}
def __init__(self, api_key: Optional[str] = None):
"""
Initialize the Claude API interface.
"""
self.api_key = api_key or config("CLAUDE_API_KEY", default=None)
if not self.api_key:
raise ValueError("API key must be provided either directly or through environment variables")
self.client = anthropic.Anthropic(api_key=self.api_key)
self.model = "claude-3-5-sonnet-20241022"
Claude Features
The ClaudeAI class provides various specialized methods:
General Text Generation
def generate_response(self,
prompt: str,
max_tokens: int = 2000,
temperature: float = 0.7,
system_prompt: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
"""
Generate a response from Claude.
"""
Chat Generation
def generate_chat(self,
messages: List[Dict[str, str]],
max_tokens: int = 1000,
temperature: float = 0.7,
metadata: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
"""
Generate a response in a chat context.
"""
Product Matching
def get_matched_products(
self,
prompt: str,
etypes: Optional[List[Dict[str, str]]] = None,
full: bool = True,
model: str = "claude-3-5-sonnet-20241022",
temperature: float = 0.7,
max_tokens: int = 1000
) -> Union[Dict[str, Any], str]:
"""
Generates product matches based on the provided prompt and parameters.
"""
Product Specifications
def get_product_specs(
self,
prompt: str,
etypes: Optional[List[Dict[str, str]]] = None,
full: bool = True,
model: str = "claude-3-5-sonnet-20241022",
temperature: float = 0.7,
max_tokens: int = 2000
) -> Union[Dict[str, Any], str]:
"""
Generates detailed product specifications based on the provided prompt.
"""
Vehicle Specifications
def get_vehicle_specs(
self,
prompt: str,
full: bool = True,
model: str = "claude-3-7-sonnet-20250219",
temperature: float = 0.7,
max_tokens: int = 4000
) -> Union[Dict[str, Any], str]:
"""
Generates vehicle specifications based on the provided prompt.
"""
Motorbike Specifications
def get_motorbike_specs(
self,
prompt: str,
full: bool = True,
model: str = "claude-3-5-sonnet-20241022",
temperature: float = 0.7,
max_tokens: int = 4000
) -> Union[Dict[str, Any], str]:
"""
Generates motorbike specifications based on the provided prompt.
"""
Product Keywords
def get_product_keywords(
self,
prompt: str,
tag_count: int = 10,
languages: Optional[List[str]] = None,
industry: Optional[str] = None,
full: bool = True,
model: str = "claude-3-7-sonnet-20250219",
temperature: float = 0.7,
max_tokens: int = 500
) -> Union[Dict[str, Any], Union[List[str], Dict[str, List[str]]]]:
"""
Generates relevant keyword tags for a given product prompt.
"""
Claude Error Handling
The implementation includes comprehensive error handling for API errors:
except anthropic.APIError as e:
raise anthropic.APIError(
request=e.request,
body=e.body,
message=f"API request failed: {str(e)}"
)
Common Use Cases
1. Product Information Extraction
Both ChatGPT and Claude can be used to extract structured information about products:
# Using ChatGPT
chatgpt = ChatGPT()
product_specs = chatgpt.get_product_specs("iPhone 15 Pro Max")
# Using Claude
claude = ClaudeAI()
product_specs = claude.get_product_specs("iPhone 15 Pro Max")
2. Vehicle Information
Claude provides specialized methods for vehicle information:
claude = ClaudeAI()
vehicle_specs = claude.get_vehicle_specs("2023 Toyota Camry")
motorbike_specs = claude.get_motorbike_specs("2023 Kawasaki Ninja")
3. Keyword Generation
Both APIs can generate keywords for products:
# Using Claude
claude = ClaudeAI()
keywords = claude.get_product_keywords(
prompt="Apple MacBook Pro 16-inch 2023",
tag_count=15,
languages=["en", "es"],
industry="e-commerce"
)
4. Color Analysis
ChatGPT can analyze colors in marketing names or images:
chatgpt = ChatGPT()
basic_colors = chatgpt.get_basic_color_matches_from_marketing_color("Starlight Gold")
image_colors = chatgpt.extract_image_colors("https://example.com/image.jpg")
Best Practices
-
Use the Right Model for the Task:
- Use Claude for detailed product specifications and vehicle information
- Use ChatGPT for image analysis and color matching
-
Handle Rate Limits:
- Both APIs have rate limits
- Implement exponential backoff for retries
- Cache results when possible
-
Control Token Usage:
- Set appropriate
max_tokensvalues based on the expected response length - Use lower values for tasks that require shorter responses
- Set appropriate
-
Temperature Tuning:
- Use lower temperature values (0.0-0.3) for factual responses
- Use higher values (0.7-1.0) for more creative or varied outputs
-
Structured Output:
- Both APIs support generating structured JSON output
- Use system prompts that clearly specify the expected format
-
Error Handling:
- Always handle potential API errors
- Implement retries with backoff for transient errors
- Provide meaningful error messages to the user
-
Response Processing:
- Parse and validate JSON responses
- Handle potential parsing errors
- Extract relevant information from the response